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Abstract 
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We introduce the notion of the stopping redundancy hierarchy of a linear block code as a measure of the trade-off between 
performance and complexity of iterative decoding for the binary erasure channel. We derive lower and upper bounds for the stopping 
redundancy hierarchy via Lovasz's Local Lemma and Bonferroni-type inequalities, and specialize them for codes with cyclic 
parity-check matrices. Based on the observed properties of parity-check matrices with good stopping redundancy characteristics, 
we develop a novel decoding technique, termed automorphism group decoding, that combines iterative message passing and 
permutation decoding. We also present bounds on the smallest number of permutations of an automorphism group decoder needed 
to correct any set of erasures up to a prescribed size. Simulation results demonstrate that for a large number of algebraic codes, 
the performance of the new decoding method is close to that of maximum likelihood decoding. 
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. weight distribution, girth, and diameter of the code graph. One exception is the binary erasure channel (BEC) transmission 



The error-correcting performance of linear block codes under iterative decoding depends jointly on a number of combinatorial 
properties of their parity-check matrices and corresponding Tanner graphs. These parameters include the minimum distance, 



scenario, where only one class of combinatorial objects, termed stopping sets, completely characterizes the failure events during 
edge removal iterative decoding [1]. 

A stopping set is a collection of variable nodes in the Tanner graph of a code such that all check nodes in the subgraph 
induced by the variable nodes and their neighbors have degree at least two. The size of the smallest stopping set, as well as the 
distribution of stopping set sizes, depends on the particular form of the parity-check matrix used for decoding. Since including 
a large number of rows in the parity-check mattix of a code ensures increased flexibility in meeting predefined consttaints on 
the sttucture of stopping sets, several authors recently proposed the use of redundant parity-check matrices for improving the 
performance of iterative decoders [2], [3], [4], [5], [6]. The effects of augmenting the sets of parity-checks in matrices from 
random ensembles were also studied in [7], [8]. 

Redundant rows in a parity-check matrix improve the performance of the decoder, but they also increase the overall time 
and hardware complexity of decoding. This motivates the study of achievable trade-offs between the number of redundant 
rows and the size of the smallest stopping set(s) in the chosen parity-check matrix. In this context, Schwartz and Vardy [2] 
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introduced the notion of the stopping redundancy of a linear block code. The stopping redundancy represents the smallest 
number of codewords that span the dual code and constitute the rows of a matrix with no stopping sets of size smaller than 
the minimum distance of the code. The same authors also derived lower and upper bounds on the stopping redundancy, the 
latter growing exponentially with the co-dimension of the code for most examples considered. This finding raised the question 
if there exist codes for which one can significantly decrease the number of redundant rows in a parity-check matrix by slightly 
decreasing the size of its smallest stopping set. 

Some results in this direction were independently derived by Weber and Abdel-Ghaffar [9], and Hollmann and Tolhuizen 
[10], who posed and partly solved the more complicated question of determining the smallest redundancy needed for decoding 
all correctable erasure patterns. 

We extend the scope of the work in [10] by introducing the notion of the stopping redundancy hierarchy of a code. The 
stopping redundancy hierarchy characterizes the achievable trade-off between the stopping distance of a parity-check matrix 
and its number of rows. We focus our attention on codes with parity-check matrices of cyclic form. In particular, we analyze 
cyclic and extended cyclic codes, which have a rich mathematical structure and large minimum distance. We also improve a 
set of bounds presented in [10], pertaining to the family of BCH codes. 

For cyclic codes, we derive the best known constructive upper bounds on the stopping redundancy hierarchy, using parity- 
check matrices of cyclic form. Our construction methods imply that, rather than adding redundant rows to the parity-check 
matrix, one can change the belief propagation algorithm instead, by combining it with permutation decoding. Permutation 
decoding was first proposed for decoding of cyclic codes over the binary symmetric channel in [11]. In the new setting, 
permutation decoders can be seen as creating "virtual" redundant rows in the parity check-matrix of a code. The permutation 
algorithm itself operates on properly designed non-redundant parity-check matrices, and it has the property that it sequentially 
moves collections of erasures confined to stopping sets into positions that do not correspond to stopping sets. 

The contributions of our work are three-fold. The first contribution consists in introducing the notion of the stopping 
redundancy hierarchy of a code and providing general upper and lower bounds on the elements of this ordered list. The 
second contribution lies in deriving upper and lower bounds on the stopping redundancy hierarchy of parity-check matrices 
of cyclic form. For the class of cyclic codes, we characterize the relationship between the stopping redundancy hierarchy and 
the minimum- weight codewords of their dual codes. As a third contribution, we demonstrate how the stopping redundancy 
hierarchy can be studied in the context of permutation decoding. More specifically, we describe a new decoding strategy, 
termed automorphism group decoding, which represents a combination of iterative message passing and permutation decoding. 
In connection with iterative permutation decoders, we study a new class of invariants of the automorphism group of a code, 
termed s-Stopping set Automorphism group Decoder (s-SAD) sets. 

The algorithms described above are tested on the [23, 12, 7] Golay code and the extended [24, 12, 8] Golay code, a set 
of primitive BCH codes, as well as on a representative subclass of quadratic -residue codes [11]. Our findings indicate that 
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automorphism group decoders exhibit near-maximum likelihood (ML) performance for short to moderate length cyclic codes. 

The paper is organized as follows. Section[n]contains a summary of the terminology used throughout the paper and introduces 
the notion of the stopping redundancy hierarchy of a linear block code. A collection of general upper and lower bounds on the 
hierarchy is presented in Section [III] Section [IV] outlines several methods for studying the stopping distance of cyclic codes. 
Section [V] introduces the automorphism redundancy, as well as the notion of s-PD and s-SAD sets. Automorphism group 
decoders are described in Section [VT] Section [VHl provides a sampling of results regarding the performance of automorphism 
group decoders. Concluding remarks are given in Section [Villi 

II. Definitions and Terminology 

We state next the definitions and terminology used throughout the paper. We restrict our attention to binary, linear block- 
codes C with parameters [n, k, d], used for signalling over the binary erasure channel (BEC). The erasure probability EP of a 
BEC is assumed to satisfy < EP < 1. The iterative edge removal decoder operates on a suitably chosen parity-check matrix 
H of the code C. For such a communication scenario, decoding errors are confined to subsets of codeword-coordinates termed 
stopping sets [1]. 

Stopping sets can be formally defined by referring either to the parity-check matrix of the code, or its underlying Tanner 
graph. For the former approach, we first introduce the notion of a restriction of a matrix. 

Definition 2.1: Let the columns of H be indexed by J = {0, . . . , n — 1}. For a set / C J, |/| > 1, the restriction of H to 
I is the matrix that consists of the set of columns of H indexed by the elements of /. 

Definition 2.2: For a given parity-check matrix H of C, a stopping set of size a is a subset / (|/| = a) of columns of H, 
such that the restriction of H to / avoids rows of Hamming weight one. 

Alternatively, let H be a fixed parity-check matrix of C, and let the bipartite graph G — (LUR, E) be such that the columns 
of H are indexed by the variable nodes in L, and the rows of H are indexed by check nodes in R. For two vertices i E L 
and j £ R, <G E iff Hji = 1. The graph G constructed in this manner is called the Tanner graph of the code C with 

parity-check matrix H. For S C L, we use r(S') to denote the set of neighbors of S in R. 

Stopping sets can be defined using the notion of Tanner graphs as follows. 

Definition 2.3: Given a bipartite graph G = (LUR, E), we say that S C L is a stopping set if the degree of each vertex 
in T(S), restricted to the subgraph induced by £ U r(S'), is greater than one. 

The stopping distance of a parity-check matrix H, and its underlying Tanner graph, is the cardinality of its smallest stopping 
set. Note that in the classical sense, the stopping distance does not actually represent a distance measure. Nevertheless, we 
adopt this terminology since it is by now standardly used in the coding theory literature. 

The sizes and the number of stopping sets in a Tanner graph depends on the particular choice of the parity-check matrix. It is 
straightforward to see that adding rows to a fixed parity-check matrix H may only increase its stopping distance. To maintain 
the integrity of the code, the added rows must represent linear combinations of the base vectors in H, and we refer to such 
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rows as redundant rows (redundant parity-checks). A parity-check matrix H containing redundant parity-check equations is 
henceforth termed a redundant parity-check matrix. The phrase "parity-check matrix" is reserved for a matrix with the smallest 
possible number of rows, i.e. for a matrix of full row-rank; similarly, the symbol m is reserved for the row-dimension of the 
(possibly) redundant parity-check matrix H. Finally, we say that a matrix H (or, a subset of rows in H) resolves a set of 
coordinates I if the restriction of H (or the subset of rows of H) to I does not correspond to a stopping set. Observe that 
resolvability does not imply that all coordinates in / can be retrieved if I is not a stopping set. In the latter case, i.e. when I 
is not a stopping set itself but "contains" one or more stopping sets, only a subset of its coordinates can be retrieved. 
We are now ready to define the stopping redundancy hierarchy of a code. 

Definition 2.4: Let C be a linear code with minimum distance d. For I < d, the £-th stopping redundancy of C is the 
smallest non-negative integer pe(C) such that there exists a (possibly redundant) parity-check matrix H of C with pe(C) rows 
and stopping distance at least I. The ordered set of integers 

( Pl (C),p 2 (C),p 3 (C),...,p d (C)) 

is called the stopping redundancy hierarchy of C. The order of an element in the stopping redundancy hierarchy is its position 
in the list. The integer pd(C) is denoted the stopping redundancy of C, and was first introduced in [2]. 

For codes with minimum distance d > 3, no two columns of the parity-check matrix are identical nor is any of the columns 
equal to the all-zero vector. Therefore, pi(C) = /02(C) = /03(C) = n — k. Consequently, only stopping redundancies of order 
larger than three are considered. Finding the stopping redundancy of a given order is very likely a complicated problem, since 
it was recently shown that computing the stopping distance of a matrix H is NP-hard and NP-hard to approximate [12], [13]. 
This is why we henceforth focus only on deriving upper and lower bounds on the elements of the hierarchy. 

In the exposition to follow, we also frequently refer to the notion of (uncorrectable erasure patterns [10], defined below. 
Only general properties of the code, but not the particular choices of their parity-check matrices, determine whether a pattern 
is correctable or not. 

Definition 2.5: Let the entries of a codeword be indexed by J = {0, . . . , n — 1}, and define the support of the codeword as 
the set of its non-zero coordinates. A correctable erasure pattern is a set I C J that does not properly contain the support of 
any codeword c e C. An uncorrectable erasure pattern is a set of positions that does not correspond to a correctable erasure 
pattern. 

It is clear from the definition of uncorrectable erasure patterns that they represent erasure configurations that cannot be 
successfully reconstructed via ML decoding. An uncorrectable erasure pattern contains a stopping set, but the converse claim 
is not true. 
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III. Bounds on the Stopping Redundancy Hierarchy 
A. Lower Bounds on the Stopping Redundancy Hierarchy 

We present next lower bounds on the stopping redundancy, £ = 4, . . . , d, of arbitrary binary linear codes. The first two 
bounds represent generalizations and extensions of lower bounds on the stopping redundancy derived in [2], For completeness, 
we briefly state this result below. 

Theorem 3.1: ([2]) Let C be a binary linear code with parameters [n, k, d], and let 

uJu = max{[(n + l)/cr] — 1, d }, a = 1, . . . , d — 1, (1) 
where d 1 - denotes the minimum distance of C^. Then 



Pd(C) > max [n-k, max ° , a € {1, . . . , d - 1}. 

It is instructive to briefly repeat the arguments leading to the results in Theorem 13.11 

The bound is established by first noting that there exist ( n J subsets of columns of H of cardinality a. A row of Hamming 
weight w can resolve exactly w ( n ~™) stopping sets of size a. A simple application of the union bound shows that 

n\ .../n — w(i)\ / ,..fn — w(i) 

ff )<x>M( ,_; ; ]<MC) T (4)( 

where u>(i) denotes the weight of the i-th row in H. The result follows by noting that this expression has to hold for all 
cr < d — 1 and that the upper bound is maximized by choosing the row weights w(i) according to Equation ([T). 
Theorem 3.2: Let C be a binary [n, fe, d] linear code, and let 

o'er = max{[(n + 1)/<t] — 1, d }, c = 1, . . . , d — 1, 

where d 1 - denotes the minimum distance of . Then 

pt(C) > max - fc, max ^ - |nL g ^ j , cr G {1, 1}. 
Note that Theorem |3.2| is a straightforward generalization of Theorem 13. II Hence, the proof of this result follows the proof of 



Theorem 13. li very closely and is therefore omitted. Both Theorem 13 . 1 1 and I3T21 can be shown to be very loose through numerous 
examples. 

Tighter lower bounds on the stopping redundancy hierarchy can be derived by means of a more precise count of the number 
of subsets of columns of H that do not correspond to stopping sets. More specifically, the number of such subsets equals the 
cardinality of the union of all subsets for which the restriction of one individual row of H does not have weight one. The 
result of Theorem 13.21 represents a weak bound on the stopping redundancy hierarchy due to the use of the inherently loose 
union bound. More accurate bounds can be calculated by invoking the principle of inclusion-exclusion, stated below. 

Theorem 3.3 ([14]): Let \A\ denote the cardinality of the set A, and let V be the set {1, . . . , |V|}. For a family of sets 
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Let E CTii denote the set of stopping sets of size cr resolved by the i-th row of H. Also, let i? = {1, 2, . . . , m} denote the 
set of row indices of H, and let Dj denote the set of all j-subsets of R. 

The number of stopping sets of size a resolved by H can be found from the PIE equation as 



where 



E S °' T - 

ted , 



1=1 ICR J =1 

Here, s^t denotes the number of stopping sets of size a resolved by the rows indexed by elements of the set T. 



There exists a simple relationship between the number of stopping sets resolved by a given parity-check matrix and the ^-th 
stopping redundancy hierarchy of the underlying code. The latter equals the smallest possible dimension to of a matrix that 
resolves all stopping sets of size a smaller than £, i.e. an to such that 



1< a < I. 



The exact calculation of the number of stopping sets resolved by an arbitrary collection of j rows is very complex, unless 
the code has certain regularity properties. One such property is that all collections of j codewords from C 1 - resolve the same 
number of stopping sets. In this case, 5 CTJ = (™) • s CT , where s CT denotes the number of stopping sets of size a resolved by 
the rows indexed with 1, . . . , j. 

The PIE is frequently applied in the form of an upper or lower bound: this is accomplished by neglecting terms involving 
intersections of sets including more than b terms, i.e. by retaining only those terms for which |/| < b [14]. Whenever the 
sign of the first omitted term is negative, the resulting expression represents an upper bound on the left hand side of the 
PIE formula. Otherwise, the obtained formula represents a lower bound. Inequalities obtained in this manner are known as 
Bonferroni inequalities. 

There exists a rich body of work on Bonferroni inequalities [14], [15], and one of its forms most amenable for stopping set 
analysis is given below. 
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This allows us to state the following result. 

Proposition 3.4: The number of distinct stopping sets of size a resolved by a parity-check matrix H with to rows satisfies 



i=l 



1=1 m i,j = l 

i < j 



(3) 
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The £-th order stopping redundancy pi(C) is lower bounded by the smallest integer m for which the right hand side of 
Equation (0) equals or exceeds (™), for all a < £. 

Proof: Equation (01 is a straightforward consequence of Equation (O; the second observation follows from the fact that, 
in order for a matrix with m rows to have stopping distance at least a, the upper bound on the number of stopping sets it 
resolves must exceed the total number of stopping sets of size a or less. ■ 
The above results will be specialized for codes with cyclic parity-check matrices in Section [IV] General expressions for the 
set intersection cardinalities of (0 are presented below. They reveal that the weights of the rows, as well as the gaps (spacings) 
between non-zero elements in the rows of a (redundant) parity-check matrix H bear a strong influence on its stopping distance 
hierarchy. 

Definition 3.5: Let O; and Z;, I £ {1, . . . ,m}, denote the sets of positions of ones and zeros in the Z-th row of the parity- 
check matrix H with m rows, respectively. The intersection number |Xj H Yj\ of H, where X and Y serve as placeholders 
for O and Z, and where i, j £ {1, . . . , m}, is defined as the cardinality of the set P C J of column positions p such that 
Hi tV = X and Hj. p = Y for all p £ P. 

Lemma 3.6: The number of stopping sets of size a, resolved by the z-th row of H of weight equals 

i„ / s / n — u)(i)\ 

|Ea,i|=wW' ( a _ 1 J' 

The number of stopping sets of size a, resolved jointly by two rows of H indexed by i and j, equals 

p.,, n s^-l = |o, n o,| ■ ('^f ') + Pi n z,\ ■ \z, n o,| • ('^f ') • 



Proof: The first part follows from the fact that a stopping set of size a can be resolved by the restriction of a row of 
weight one, with the one-entry chosen among oj(i) options and the remaining zeros from ("~"^) choices. The second claim 
is a consequence of the following observation: a pair of rows resolves the same stopping set if either both rows share a "1" 
and all the "0"s in the remaining positions of the stopping set; or, if within the support of the stopping set the two rows share 
all except for two positions for the "0" symbols, and have two non-overlapping positions with the symbol "1". ■ 
The expressions in Lemma (13.6b can be further simplified by assuming that all rows in H have weight u> (provided that C 1 - 
has sufficiently many codewords of weight w), and by writing 



|Ep| = max |E CT ,t n Eo-jj 



Under these assumptions, 



where |Ep| can be found based on the formula in Lemma [3~6l 



E«r,i|<m.«- L , --• 9 ) -|*H (4) 
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> max 



(5) 



A lower bound on the £-th stopping redundancy can be obtained from Lemma [3~6l by observing that the upper bound in (01 
has to exceed (") for all a < £ . As a consequence, 

" - 1^1 

(T-i) l=P 

where the qualifier ui(i) — ui denotes that a constant codeword weight u> was assumed for this bound. The previous analysis 
shows that, in order to resolve a large number of stopping sets by a (redundant) parity-check matrix H with a small number 
of rows, one has to minimize the number of stopping sets resolved jointly by subsets of rows. Therefore, small sizes for the 
intersections of the supports of pairs, triples, etc., of rows are desirable. This is why we henceforth restrict our attention to 
matrices H that contain minimum weight codewords of C 1 - as their rows. 

It is also worth pointing out that for many codes the codewords of minimum weight or other fixed weight represent a design 
[11]. In those cases, the intersection numbers of the minimum weight codewords can be obtained from the parameters of the 
design. Examples supporting this observation, pertaining to Quadratic Residue (QR) codes and the Golay codes [11], are given 
in subsequent sections. 

Example 3.1: Consider a parity-check matrix of the form shown below: 



H 



1 1 
10 10 



The columns indexed by {1, 2, 3} have a unique row-restriction, namely "100". The columns indexed by {2, 4, 5} and {3, 4, 5} 
have row restrictions "001" and "010", respectively. Consequently, both rows simultaneously resolve stopping sets indexed 
by the described set of columns. More precisely, both rows resolve the three restrictions indexed by {1,2,3}, {2,4,5}, and 
{3, 4, 5}, while all other restrictions are resolved by at most one of the rows. 

Example 3.2: Consider the family of [2 s — 1, s, 2 S_1 ] simplex codes. Since simplex codes have constant codeword weight 
2 S ~ 1 , for all distinct indices i and j, the intersection numbers are given as \Oi <~)Oj\ = 2 S ~ 2 , \Oi fl Zj\ = \Zi P\Oj \ = 2 S ~ 2 , 
and \Zi n Zj\ = 2 s — 1 — 3 ■ 2 S ~ 2 = 2 s-2 — 1. The bound in © shows that for i ^ j, the dual code of a simplex code, a 
[2 s — 1, 2 s — s — 1, 3] Hamming code C(s), satisfies 

'2 s - 2 - r " 



s CT ,ins„ 2 



■is-2 



2 s-2 _ l 

a -I 



cr-2 



and that, consequently, 



Ps(C(s))| u 2 .-i > max 

<T<0 



Simple evaluation of the bound reveals that 



f; 1 ) - 2 s - 2 ■ 


V <j- 






2 s " 1 ■ (V, 1 ! - 2- 


-2 . 


(V, 1 ) MV 




> max ^[3- 2 2 


"1> 






1-2 1 -* J 1 





which implies the trivial result p^(C{s)) > 3, s > 1. This illustrates the fact that PIE-based lower bounds can be arbitrarily 
weak, since it is known that p 3 (C(s)) = s. 
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B. Upper Bounds on the Stopping Redundancy Hierarchy 

Upper bounds on the stopping redundancy hierarchy can be derived by invoking probabilistic methods [16]. Our derivations 
follow the framework we developed in [7], based on Lovasz Local Lemma [17], [16]. Related probabilistic techniques were 
also used for deriving upper bounds on the stopping redundancy in [4]. 

For subsequent derivations, we need the following results, known as Lovasz Local Lemma (LLL), as well as the high- 
probability variation of LLL. 

Lemma 3.7: Let E±, E2, ■ ■ . , En be a set of events in an arbitrary probability space. Suppose that each event Ei is 
independent of all other events Ej, except for at most r of them, and that 

P{Ei} <p, VI < i < N. (6) 

If 

ep(r + l)<l, (7) 
where e is the base of the natural logarithm, then P{f]f =1 Ei} > 0. 



P{Ei}<-L(l-±) ,l<i<N, (8) 



Furthermore, let < e < 1. If 

PiEA < 

N V TV 
*en P{CCi Ei}>l-e. 

Based on Lovasz Local Lemma © and its high-probability variation ([8]), one can obtain the following bounds on the £-th 
stopping redundancy of a code, for I < [2±1 J . The derivations are straightforward, and based on associating stopping distance 
properties of restrictions of a parity-check matrix with the events Ei described in the statement of LLL. 

Theorem 3.8: Let C be an [n, k, d] code. If I < [^J and 

l-i 



i+iogE(G)-(T) 
.7=1 x 



m> ^— + n -k-£ + l, (9) 

- log (1 - 2^t) 

then pi(C) < 771. Alternatively, if the conditions of the theorem are fulfilled, then there exists at least one parity-check matrix 
with m rows and stopping distance at least I. 

The proof of this non-constructive bound is given in Appendix Q] An asymptotic estimate of the above bound is presented in 
Appendix HTl for codes with d = const. • n. 

One way to make the finding of Theorem 13.81 more useful for practical purposes is to generalize it by invoking the high 
probability version of LLL, as stated below. 

Theorem 3.9: Let C be an [n, k,d] code, let I < [^J, and assume that m fulfills the condition 

log + ( E [(?) - (V) - il 1 • (1 - 



777 > 
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Then the probability that a parity-check matrix consisting of m randomly chosen codewords of the dual code (with possible 

repetitions of a codeword) has stopping distance I is at least 1 — e. With at most n — k — i + 1 additional rows, the matrix 

also has rank n — k and represents a valid parity-check matrix of the code C. 

The proof of the theorem is given in Appendix Hill Numerical results for a selected set of values of I is given in Section HVl 
As the results of Theorems 13.81 and 13.91 are non-constructive and conditioned on I < L^"J' we present another method 

for finding upper bounds on the £-th stopping redundancy. This second class of upper bounds on the stopping redundancy 

hierarchy of a code is constructive in nature, and based on the following result. 

Theorem 3.10: ([2]) Let C be a binary linear code with parameters [n, k, d], with d > 3. Then 

fn — k\ fn—k\ 

mc)<( 1 )+...+ [ d _ 2 y do) 

The proof of the bound in Theorem 13. 101 is constructive: one starts with an arbitrary parity-check matrix H of the code C, 
and then successively adds all sums of not more than d — 2 distinct rows of H . 

It is straightforward to invoke Theorem |3.10| for upper-bounding the stopping redundancy hierarchy pi(C) of a code - more 
precisely, in terms of adding all sums of at most I — 2 rows to a given parity-check matrix. This result is stated below. 

Theorem 3.11: Let C be a binary linear code with parameters [n, k, d), with d > 3. Then 

/ n — k\ fn — k\ 

MC)<{ 1 )+-+{ e _ 2 } (") 

Assume that one can identify a sub-code B of C with dual distance dg. Since the generators of a sub-code form a subset of 
the generators of C , one needs to apply the procedure of adding redundant rows leading to Theorem 13. II or Theorems 13.813.91 
only to the basis vectors in B in order to ensure that the redundant matrix has stopping distance at least dg. This argument 
leads to the following result. 

n 

Theorem 3.12: Let 9 be the set of all sub-codes of the dual code C of a linear [n, k, d] code C that have support weight 
n and dual distance I. Furthermore, let the dimensions of the sub-codes in 8 be Ki, i — 1, . . . , |0|, and define K = mini K{. 
Then 

MC)<(f)+...+ ( £ X 2 ). (12) 
Proof: Let C\ be an [n, n — Ki,£] code, and let its dual code be a sub-code of C of support n and dimension K\. 

From Equation CEE M C < (^) + • • • + G-a)- As C i 

is a sub-code of C 1 -, the rows of the parity-check matrix of C\ are 
a subset of the rows of the parity-check matrix of C. As a result, the upper bound also holds for pe(C). ■ 
Example 3.3: Let Q be the [24, 12, 8] extended Golay code. The code Q is self-dual and contains a [24, 10, 8] sub-code. 
This code is unique [18], and has minimum dual distance 6. Therefore 

»<« <- (?) + e») + p + n =3*, 

This bound is not tight as it by far exceeds the best known upper bound (found through extensive computer search [2]) on 
Ps(G) < 34. 

1 The support weight of a sub-code of a code is defined as the number of positions for which at least one of the codewords of the sub-code is non-zero. 
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Example 3.4: Let Cbch.i be the [31, 16, 7] BCH code. Since BCH codes are nested, this code is a sub-code of a [31, 21, 5] 
BCH code Cbch,2- Consequently, C"b CH 2 C C^ cli x , so that 

IV. Case Study: Parity-Check Matrices of Cyclic Form 

The lower and upper bounds on the stopping redundancy hierarchy, presented in the previous section, hold for all linear block 
codes. Unfortunately, as illustrated with several examples, these bounds tend to be very loose. As will be shown in this section, 
much tighter upper bounds on the stopping redundancy hierarchy of certain classes of codes can be obtained constructively, 
by focusing on special forms of parity-check matrices. In particular, we consider parity-check matrices of cyclic form. Upper 
bounds on the stopping distance of matrices in cyclic form also represent upper bounds on the stopping redundancy hierarchy 
of the codes. This claim is not true for lower bounds on the stopping distance hierarchy, although the derived bounds still offer 
valuable insight into the stopping set properties of matrices in cyclic form. 

We start by specializing the bound of Section [Til] to cyclic parity-check matrices, and then proceed with a case study of 
BCH codes and codes based on cyclic difference sets. 

A. Stopping Sets in Parity-Check Matrices of Cyclic Form 

We start this section by introducing cyclic codes and cyclic parity-check matrices. 

Definition 4.1: Let C be an [n, k, d] binary linear code. A code is called cyclic if a cyclic shift of a codeword c S C is also 
a codeword. A (redundant) parity-check matrix of a cyclic code is said to be of cyclic form if it consists of m cyclic shifts of 
one given codeword of the dual code, and provided that it has row-rank n — k. 

Observe that parity-check matrices of cyclic form necessarily satisfy n — k < m < n. Also, note that a code can have a 
parity-check matrix of cyclic form without being cyclic - nevertheless, we focus our attention exclusively on cyclic codes. A 
standard form for H with m = n — k cyclic row-shifts is shown below. 

/ 1 ... 1 0\ 

1 ... 1 

1 ... 1 

H = 1 ... 1 

: '•• '•• '•• 

\o o o o o i ... i y 

A binary cyclic code can be completely characterized by a normalized parity-check polynomial h(x) = ho + h\x + h^x 2 + 
• • • + hk-ix k ~ 1 + x k , of degree k, for which hi 6 Fa, V« G [0, k — 1]. In this case, a cyclic parity-check matrix H of the 
code can be constructed by setting the entry in the i-th row and j-th column to 

^(k—j+i mod n) 



H[i,j] = 



(k — j + i modn)! 



x=0 
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where h^(x) denotes the £-th derivative of h, 1 < i < m, 1 < j < n. This leads to a parity-check matrix of the form 



Definition 4.2: Let C be a cyclic [n, fc, c?] linear code. Partition the set of codewords of the dual code C into cyclic orbits 
of its codewords. More precisely, let C be the disjoint union (J G c in which c 6 C , G c = {ir l c, j = 1, . . . , n}, and where 
7r denotes the right-cyclic shift permutation of the symmetric group S n . A designated element of G c is henceforth referred to 
as the cyclic orbit generator (cog) of G c . Whenever apparent from the context, the reference to G c will be omitted. 

Note that, in general, the first row of a redundant parity-check matrix does not have to be defined by a parity-check 
polynomial. The first row can be any cog which has the property that a sufficient number of its cyclic shifts generates the dual 
code. In what follows, with a slight abuse of notation, we use h to denote such a cog (i.e. the first row of a cyclic parity-check 
matrix H), and let h[i] denote its i-th entry. 

We start by providing an intuitive explanation why (redundant) parity-check matrices of cyclic form have good stopping 
distance properties. 

Definition 4.3: Without loss of generality, assume that the first row in a cyclic parity-check matrix H has a non-zero symbol 
in its first position. The span of the first row, denoted by u, is the largest value of the index j for which H[l,j] = 1, The 
zero-span z of h represents the number of zeros trailing the last non-zero entry. Clearly, u + z = n. 

Consider an arbitrary (redundant) cyclic parity-check matrix if of a code C. The matrix consists of a non-zero cog and 
m — 1 consecutive cyclic shifts thereof, where n~ k < m < n. The matrix H has i—1 leading zeros and z — i + 1 tailing zeros 
in row i, 1 < i < m. For any stopping set with largest column index r < z + 1, the r-th row of the matrix has an entry "1" in 
column r, and zeros in all the remaining positions confined to {1,2,. . . ,r— 1}. Therefore, the rows of H resolve all stopping 
sets of size a with support contained in the set {1, 2, . . . , z + 1}. Similarly, for all stopping sets with coordinates confined to 
the set {n — z, n— z+ 1, . . . , n}, let the smallest non-zero coordinate in the stopping set be indexed by /. In this case, the row 
indexed by / — (n — z) has a "1" in the leftmost position I, and zeros in all remaining positions of the stopping set. Therefore, 
the rows of H also resolve all stopping sets of size a with support contained in the set {n — z, n — z + 1, . . . , n}. 

It can also be easily seen that a redundant cyclic parity-check matrix with m = n rows resolves all stopping sets with 
support confined to z + 1 consecutive column indices. This straightforward analysis indicates that parity-check matrices of 
cyclic form may have good stopping distance properties. We explore these questions in more detail in the next section. 

B. Lower Bounds on the Stopping Distance of Cyclic Parity-Check Matrices 

We describe next how to bound the stopping distance of cyclic parity-check matrices by specializing the results of Section 



Proposition 4.4: The number of stopping sets of size a resolved by a cyclic (redundant) parity-check matrix with dimensions 
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m x n, n — k < m < n, is bounded from above by 



m— 1 



m (S^xl ^ (m - ^ISg-.l H S g -,((l+K)mod* m)|- 



(13) 



K=l 



The notation mod* is reserved for the modulo function, for which m mod* to equals m, rather than zero. 

In the proposition, it is tacitly assumed that the rows of the parity-check matrix are arranged in such a way that the row 
indexed by ((k + 1) mod* to) is the n-th cyclic shift of the first row. 

The result of Proposition 14.41 is a specialization of Equation ©. In a cyclic parity-check matrix, each row resolves the same 
number of stopping sets as the positions of the resolved stopping sets are cyclic shifts of each other. Furthermore, the number 
of stopping sets simultaneously resolved by two rows indexed by i and j, i ^ j, only depends on the their mutual cyclic shift 
distance k. 

Let the set XY K (h) denote all pairs of positions (a, a + k) mod* n in h, 1 < a < n, with entry X at position a mod* n 
and entry Y at position a + n mod* n, where X and Y again serve as placeholders for Z and O. For parity-check matrices of 
cyclic form, the cardinalities of these sets equal the intersection numbers given in Definition [33] i.e. = |X amo d* n H 

^a+K mod* n|, Va G {1, . . . , n}. If clear from the context, we omit the reference to h. 

Lemma 4.5: Let S[k] be the indicator function, such that 5[k] = 1 iff k = 0. The number of stopping sets resolved jointly 
by two rows of a cyclic parity-check matrix n cyclic shifts apart equals 



mod* m) I 



\OOJ ■ 



\zz K \ 

a- 1 



\OZJ ■ \ZOJ 



\zz K \ 

0--2 



where 



\oo K \ = J2 



n 

v\j-i]+s\j 



«]) 



\zz K \ = J2 



1 n 



3=1 



\zo K \ = \oz K \ = J2 



1=1 



J2(h\j] (8\j-i] + 8\j-i-K]))-2 



and 



|S<T,l| 



■13 



Note that in all equations above it is assumed that Qj = for i < j. 

The expressions in Lemma 14.51 are derived by simple counting arguments, details of which are omitted. Although the 
expressions above cannot be used to directly characterize the stopping redundancy hierarchy, they represent a useful tool for 
evaluating the stopping distance properties of cyclic parity-check matrices, as illustrated by the examples that follow. 
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Example 4.1: Consider the class of quadratic residue (QR) codes with prime length n = 3 (mod 4). Let the redundant 
parity-check matrix be of cyclic form, and let the first row of the parity-check matrix be defined by the idempotent of the code 



Q 1 - (defined in [11]), I q (x) = 1+2 x v . Here, M denotes the set of quadratic non-residues in F 



The cyclic parity-check 



matrix generated by to < n cyclic shifts of the idempotent has S a p stopping sets resolved by all pairs of rows, where 

n+1 ( ( \ n + l 



S a .2 — 



71 — 3 

4 

a - 1 



re— 3 
(7-2 



4 \\o-\) 4 

Be reminded that S a ,2 denotes the number of stopping sets resolved by all pairs of two rows (cf. Section IIII-Ab . As a 
consequence, for such a cyclic parity-check matrix to have stopping distance at least I, its number of rows \in must satisfy the 
following inequality 



pi(C) > max 

cr<£ 



C) - M 



2 \a-l) 



M 



where 



M = 



n+1 



n— 3 
4 

(7 - 1 



n + 1 



n-3 
(7-2 



A detailed derivation of this result can be found in Appendix IIVI Numerical values for the numbers pi, for three chosen QR 
codes, are given in Table U The bounds in the table imply that for the [47, 24, 11] code, the stopping distance of any redundant 

TABLE I 

Lower bounds on fi e for parity-check matrices of cyclic form generated from the idempotent of QR codes. 





fl 4 fl 5 He fPr M8 M9 MlO Mil 


[23, 12, 7] Golay 
[31,16,7] BCH 
[47, 24, 11] QR 


4 6 10 19 
4 6 9 17 

4 6 9 15 27 55 117 265 



parity-check matrix of cyclic form, defined by the idempotent, cannot exceed seven. 

Example 4.2: As a second example, consider the class of cyclic difference set (CDS) codes [11], [19], [20], formally defined 
below. 

Definition 4.6: Assume that all calculations are performed modulo n. An (n, k, A) CDS is a set Q = {do, d\, . . . , dj-i} of 
J integers with the property that any non-zero integer i < n, is a difference of two elements in Q and that there exist exactly 
A ways to choose these two elements of Q. 

For a CDS, the sets Q r — {do + r, d±+r, . . . , dj-i +r} for all r = 0, . . . , n— 1, form a cyclic 2— (n, k, A) block design [21]. 

According to [19], a CDS code of length n is characterized by a polynomial z(x) — 2~2^Zo x<li mat gi ves ri se t0 a parity-check 
polynomial h(x) = gcd(z(x),x n + 1). 

For this particular case of cyclic codes, one can show that if h(x) = z(x), then 



\OO k \=A, \OZJ 



\zo K 



k-X, \ZZ K \ =n-2k + \ V k>0, 



and that 



v o 1'n — 2k + A 
+ (fc-A) 2 



(7-2 



2 For more details regarding QR codes, the interested reader is referred to Section 16 of [11]. 
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As a result, 

- M 



lie > max 



k ■ (-:*) - m 



where 



M = A.( n ; 2 _ fc i +A ) +(fc -A) 2 .( n ; 2 _ fc 2 +A 



and where [ig is defined as in the previous example. 

Consider the class of Singer difference sets [21], with parameters 

n, k, A) = — , -, — 

V 9- 1 9- 1 9- 1 

where we specialize a = 2, and q = 2 s . The resulting difference set has parameters (n, fc, A) = (2 S (2 S + 1) + 1, 2 s + 1, 1), 
and leads to a [2 S (2 S + 1) + 1, 2 S (2 S + 1) - 3 2 , 2 2 + 2] CDS code, for which, if h{x) = z{x) as defined above, 

^•2 2a +2 s + l-j _ ^2 2s -2 s ~j _ 22s^2 2s -2 s ~j 



ui > max 



(2 s + 1) • C\) - ( 22 ;rD - 2- • ( 22 ;rf) 



The results presented in this section characterized the smallest number of rows in a cyclic parity-check matrix needed for 
resolving all (or a certain number of) stopping sets of a given size. To obtain upper bounds on these numbers, we resort 
to constructive methods that rely on identifying good choices for a cog defining the parity-check matrix. Our cog selection 
procedure, along with a comparison of all constructive redundancy parity-check design methods discussed, is presented in the 
next section. 

C. Upper Bounds on the Stopping Distance of Cyclic Parity-Check Matrices 

In what follows, we focus on deriving upper bounds on the stopping distance of cyclic parity-check matrices of the Golay 
code and BCH codes. We review several known approaches for constructing redundant parity-check matrices for Hamming 
codes with a prescribed number of stopping sets of a given size, and then compare them to those based on optimizing cyclic 
parity-check matrices. For Hamming codes, the number of stopping sets of a full-rank, non-redundant parity-check matrix 
was derived in analytical form in [3]. We observed that this number does not depend on the form of the parity-check matrix. 
However, if redundant parity-check matrices are used, the number of stopping sets clearly depends on the chosen form of the 
matrix. 

As will be shown, for a given stopping set distribution, cyclic parity-check matrices for the Golay and BCH codes offer 
row-redundancies comparable to those achievable by the best known methods described in [2] and [10], respectively. For the 
general class of BCH codes, we first provide a new method for constructing redundant parity-check matrices with prescribed 
stopping distance, by extending the work in [10]. We then compare these findings with the stopping redundancy parameters 
obtained from parity-check matrices of cyclic form. 
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1) The Binary Golay Code: To evaluate the performance of cyclic parity-check matrices of the [23, 12,7] Golay code Q, 
the 506 codewords of minimum weight eight in the dual [23, 11,8] code Q 1 - are grouped into cog orbits. The codewords of 
weight eight in Q 1 - also belong to the code Q, and can be arranged into 22 cog orbits with 23 codewords each. 

Different choices for a cyclic parity-check matrix-defining cog result in different stopping distance bounds. We concentrate 
on two cogs that offer the best and the worst stopping distance properties among all investigated cogs, and refer to them by 
the subscripts A and D. 

The cog indexed by A has the following octal representation cog[ 2 3 12 ] a — [2 1 2 1 3 5 0], with the most significant bit on 
the left side. Its polynomial representation is 1 + x 4 + x 6 + x 1Q + x 12 + a; 13 + x 1A + x 16 . The cog indexed by D has the 
representation cog[ 2 3 l x2],D = 

[34603200]. 

Upper bounds U(pj(C)) on the stopping redundancy hierarchy obtained from cogs cog[ 2 3 i i2],A' c °g[23,i2],B' an d c °g[23,i2].D 
of the parity-check matrix of the [23, 12, 7] Golay code and the general upper bounds obtained from Theorem 13 . 91 with e = 10~ 3 
and from Equation (TTOb are listed in Table [XT] 

TABLE II 

Upper bounds on the stopping redundancy hierarchy of the [23, 12, 7] Golay code. 





U(p 4 (C)) 


U(P5(C)) 


U{P6{Q) 


U(P7(Q) 


co §23,12,A 


n — k 


16 


18 


23 


c °g23,12,B 


13 


15 


19 


23 


c °g23,12,D 


n — k 


16 


21 


> 23 


Minimum 


n — k 


15 


18 


23 


Upper bound acc. to Theorem 13.91 e = 10~ J 


39 








Upper bound acc. to Eq. (10) 


1023 


1023 


1023 


1023 



As can be seen from Table |IIJ the stopping redundancy hierarchy of the Golay code is bounded as 

p 4 (C) <n~k, P5 {C) < 15, p 6 (C) < 18, P7 {C) < 23. 

In addition, from the same table it can be seen that the bound in Equation ( TTOb is extremely loose - significant reductions are 
possible when using the constructive cyclic matrix design approach. 

2) The Hamming Codes: Several approaches for generating parity-check matrices of Hamming codes that resolve correctable 
erasure patterns up to a given size were recently described by Weber and Abdel-Ghaffar [9] and Hollmann and Tolhuizen [10]. 
In [9], the authors introduced the notion of three-erasure correcting parity-check collections, capable of resolving the largest 
possible number of erasure patterns of weight three. These results were extended in [10], where generic (to, a) erasure 
correcting sets Afn,s were defined as 

{a = (ai, a 2 , . . . , Om) | ai = 1, wt(a) < a}. 

In this context, if H represents a parity-check matrix of dimension fhxa, the collection of parity-checks {aH \ a G A rTlS } 
resolves all correctable erasure patterns up to size a. 
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For the class of [2 s — 1, 2 s — s— 1, d = 3] Hamming codes, according to [10], we first construct parity-check matrices that are 
generic (m, a) erasure correcting sets, and then compare the stopping set distribution of these matrices with that of optimized 
cyclic parity-check matrices. As for Hamming codes p2(C) — /33(C) = n — k and d = 3, our comparison is performed with 
respect to the number of unresolved stopping sets of size a = 3, for several fixed values of m. 

Assume that the parity-check matrix of a Hamming code is given by a row vector over the finite field F n+ i, n = 2" — 1, n' > 
0, namely 

H h =( a o b a l b ... a^ b ), 

where a denotes a primitive element of the underlying field. Henceforth, we set b = 1. In order to form a binary parity-check 

matrix for the Hamming code, the elements of are represented as binary vectors over the vector space p l ° g ^ n+1 ^ j e 

each element a* , f £ {0, . . . , n — 1}, is described by a binary column vector. Let us refer to this parity-check matrix as the 

standard parity-check matrix for Hamming codes and denote it by H\ iyS . 

Start by constructing a generic erasure correcting set Am,u with fn = log2(?i + 1) and a = 3. By applying the generic 

erasure set construction to J?h,s> we arrive at a redundant parity-check matrix of the code denoted by H\ l g . This matrix has 

* ^{fh-l\ , . /log 2 (n + l)-l\ 

"* =E( i ) =log 2 (" + l)+ [ 2 J 

rows. Note that the only unresolved stopping sets of size three in such a parity-check matrix are the actual codewords of the 
Hamming code. We then proceed by constructing redundant parity-check matrices of cyclic form, obtained from m* consecutive 
cyclic shifts of a minimum weight codeword of the dual code. Table Hill and Table HVl show the number of unresolved stopping 
sets of size three in the redundant parity-check matrices described above, for n = 63 and n = 127. The cog vectors of the 
two cyclic parity-check matrices are 

cog [6357] = [4 142475071 13354653740], 

and 

c °g[i27,i20] = [1046135330146516366412575121561770357131 10 0], 
respectively. The vectors are listed in octal form, with the most significant bit on the left hand side. 

TABLE III 

Number of unresolved stopping sets (<t = 3) in the [63, 57, 3] Hamming code. 



m 


Number of unresolved stopping sets, a = 3 


Parity-check matrix in [10] 


Cyclic parity-check matrix 


6 = n — k 


2261 


2261 


16 


651 


655 


17 




653 


18 




651 



As one can observe from Tables ITTTI and IIVI cyclic parity-check matrices can achieve almost identical performance to the one 
offered by redundant parity-check matrices specialized for the Hamming codes. For example, there are 11970 stopping sets of 
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TABLE IV 

Number of unresolved stopping sets (<t = 3) of the [127, 120, 3] Hamming code. 



m 


Number of unresolved stopping sets, a = 3 


Parity-check matrix in [10] 


Cyclic parity-check matrix 


7 = n - k 


11970 


11970 


22 


2667 


2672 


26 




2667 



size a — 3 in a full-rank parity-check matrix of the [127,120,3] Hamming code. This number can be reduced to 2667 stopping 
sets only, if one uses 22 rows of the construction described in [10], or 26 rows, when using parity-check matrices of cyclic 
form. These findings motivate a further study of redundant cyclic parity-check matrices, and we describe some illustrative 
examples pertaining to the family of BCH codes in the next section. 

3) Cyclic Parity-Check Matrices of Double- and Triple-Error Correcting BCH Codes: We present constructive upper bounds 
on the stopping redundancy hierarchy of the [127,113,5] double-error correcting BCH code, and the [31,16,7] triple-error 
correcting BCH code. Similarly as for the Golay code, we only consider minimum-weight codewords of the dual code for 
the purpose of generating redundant parity-check matrices in cyclic form [22]. In particular, we focus on four selected cogs, 
indexed by A through D, and analyze their underlying parity-check matrices in detail. 

The dual of the [127, 113, 5] BCH code has 4572 codewords of minimum weight 56, which can be separated into 36 cog 
orbits. The octal representations of four different cogs used in our constructions are listed below: 

c °g[i27.ii 3 ].A = [176403065445407504547651616020426524244005 6], 
cog[i27,ii 3 ] )B = [1724250261215411115261072125516140465414274], 
c °g[i27,ii3],c = [17526553364613126421071 1704024025403045224 2], 

c °g[i27.ii 3 ].D = [1751703125267346502102070365406122101430644]. 

The dual of the [31, 16, 7] BCH code has 465 codewords of minimum weight eight, which can be partitioned into 15 cog 
orbits. Four representative cogs, indexed by A through D, are listed below. 

cog [3 i,i6],A = [14140 5 000 2 2], 
cog [3 i,i6],B = [140 610410 20], 
cog [3 i,i6],c = [15 000 5 00414], 

cog [3 i,i6],D = [15040 200130]. 

We now compare the stopping distance properties of parity-check matrices of cyclic form with that of a novel construction 
of redundant parity-check matrices generalizing the method in [10]. In a nutshell, the construction exploits the fact that BCH 
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codes are nested, and that they are sub-codes of Hamming codes. Redundant parity-check equations are chosen in two different 
steps: in the first step, they are selected to eliminate stopping sets corresponding to certain codewords of the Hamming code, 
while in the second step they are chosen to eliminate stopping sets that do not correspond to codewords in the Hamming 
code. A detailed description of this scheme is given in Appendix [V] For simplicity, we refer to it as the generalized HT 
(Hollmann-Tolhuizen) construction. 

We next compare the stopping distances of various redundant parity-check matrices of the aforementioned BCH codes. The 
smallest values found by this comparison represent constructive upper bounds on pe(C), I < d, of BCH codes. 

Table M lists the number of rows in a redundant parity-check matrix needed to achieve a given stopping distance, for matrices 
constructed by the generalized HT method, the upper bound from [10], and a bound obtained from cyclic parity-check matrices. 
In addition, the upper bound from [2], given in Equation ( [Tol l, is also shown. All results pertain to the [127, 113, 5] BCH code. 

TABLE V 
Upper bounds on pe(C), I < 5. 



Code 


U{ P 4{Q) 


Cyclic 


Gen. HT 


L10J 


General [2] 


[127, 113, 5] 


20 


96 


92 


469 


Code 


U{ P5 {Q) 


Cyclic 


Gen. HT 


L10J 


General [2] 


[127, 113, 5J 


34 


229 


378 


469 



The values presented for cyclic codes are obtained by searching for the best cog orbit, with cogs corresponding to minimum 
weight codewords. The cog indexed by A has the "best" stopping set properties, when both the numbers of stopping sets of 
size three and four are considered. Stopping distances of all four described cogs are given in Table [Vl] 

As a consequence, for the [127, 113,5] BCH code £>[i27, 113.5] one has 

P4(S[127,113,5]) < 20, P5(B[127,113,5]) < 34. 

Table [Vl shows that the general bounds derived in [2] are very loose. Although the approach from [10] provides tighter 
bounds for this class of codes, it is still significantly outperformed by both the generalized HT method, as well as the cyclic 
(redundant) parity-check matrix construction. 

TABLE VI 

Upper bounds on the stopping redundancy hierarchy of the [127, 113, 5] BCH code. 





U{p 4 {Q) 


U(p 5 (C)) 


co gl27,113,A 


20 


34 


co gl27,113,B 


22 


34 


co gl27,113,C 


23 


46 


co gl27,113,D 


22 


45 


Minimum 


20 


34 


Eq. not 


469 


469 



The results of a similar analysis performed on the [31, 16, 7] BCH code Z?[ 31 16 >7 i are shown in Table [VHl As can be seen 
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TABLE VII 

Upper bounds on the stopping redundancy hierarchy of the [31, 16, 7] BCH code. 





U(p 4 (C)) 


U{ P5 (C)) 


U{ P6 (C)) 


U{pr(C)) 


c °g31,16,A 


n — fc = 15 


18 


19 


21 


c °g31,16,B 


n — k = 15 


16 


20 


22 


c °g31,16,C 


n — k = 15 


n — fc = 15 


20 


28 


C °S31,16,D 


n — k = 15 


16 


21 


26 


Minimum 


n — k = 15 


n — fc = 15 


19 


21 


Upper bound acc. to Theoreml3.9l e = lO" 13 


45 








Upper bound acc. to Eq. (101, ill) 


91 


175 


4943 


4943 



from the table, the stopping redundancy hierarchy of the [31, 16, 7] BCH codes is upper bounded as 

Pi (#[31,16,7]) = 15 ' P5 (£[31,16,7]) = 15 > P6 (£[31,16,7]) < 19 : P7(£[31,16,7] ) < 21. 

V. The Automorphism Redundancy 

One important observation motivates our subsequent study of stopping redundancy properties of cyclic (redundant) parity- 
check matrices. It is the simple fact that using a collection of cyclically shifted rows in H for resolving a stopping set has 
the same effect as using only one of these rows and then cyclically shifting the received codeword. 

In general, the same observation is true for parity-check matrices that consist of rows that represent different images of one 
given row-vector under a set of coordinate permutations. Such coordinate permutations must necessarily preserve the codeword 
structure, i.e. they must correspond to automorphisms of the code. Furthermore, if properly chosen, such automorphisms may 
be used to relocate the positions of the erasures in the received codeword to coordinates that do not correspond to stopping 
sets. 

We propose to use this observation in order to construct parity-check matrices of codes that, jointly with a new decoding 
technique, allow the edge-removal algorithm to avoid detrimental effects of stopping sets on its performance. For this purpose, 
we first recall the definition of the automorphism group of a code. 

Definition 5.1: ([11, Ch. 8]) The set of permutations which send a code C into itself, i.e. permutations that map codewords 
into (possibly different) codewords, are called the automorphism group of the code C, and are denoted by Aut(C). 

It is straightforward to see that Aut(C) = Aut^C^), a fact that we exploit in our subsequent derivations. 

Decoding procedures that use information about the automorphism group of a code have a long history [11, Ch. 16]. 
Such procedures are known as permutation decoding algorithms and they are traditionally restricted to decoding of messages 
transmitted over the binary symmetric channel (BScj^. Within the framework of permutation decoding, a codeword c S C, 
corrupted by an error vector e = [eo, e\, . . . , e„_i] of weight less than or equal to i, where 2t + 1 < d, is subjected to the 
following process. First, a parity-check matrix of the code of the form H = [A\I], along with the corresponding generator 
matrix G = [I\A T ], is chosen. Here, J is used to denote the identity matrices of order n — k and k, respectively. Then, the 
syndrome of the received vector y = c + e, z = Hy T , is computed. If the weight of z is greater than t, the vector y is 

3 Recently, permutation decoders were also used for decoding of messages transmitted over the AWGN channel [23], although not for the purpose of finding 
error-free information sets nor for the purpose of eliminating pseudocodewords such as stopping sets. 
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permuted according to a randomly chosen automorphism. This process is repeated until either all automorphisms are tested 
or until the syndrome has weight less than or equal to t. In the former case, the decoder declares an error. In the latter case, 
all decoding errors are provably confined to parity-check positions, so that decoding terminates by recovering the uncorrupted 
information symbols. 

For the purpose of permutation decoding, one would like to identify the smallest set of automorphisms that moves any set 
of not more than t positions in {0, . . . , n — 1} into the parity-check positions {k, k + 1, . . . , n — 1} of [A\T\. 

Definition 5.2: ([24]) If C is a t-error correcting code with an information bit index set 1 and a parity-check index set V, 
then a PD(C)-set (permutation decoding set of C) is a set S of automorphisms of C such that every t-set of coordinate positions 
is moved by at least one member of S into the check-positions V. For s < t, an s-PD(C)-set is a set of automorphisms of C 
such that every 6-set of coordinate positions, for all b < s, is moved by at least one member of S into V. 

Throughout the remainder of this paper, we will be concerned with PD and s-PD sets of smallest possible size, and we 
simply refer to them as PD sets. Clearly, PD and s-PD sets may not exist for a given code, and complete or partial knowledge 
about PD sets is available for very few codes [24]. Nevertheless, even this partial information can be used to derive useful 
results regarding the analogues of PD sets for iterative decoders operating on stopping sets. 

We introduce next the notion of a Stopping Automorphism Group Decoding (SAD) set, a generalization of the notion of 
a PD set for the edge-removal iterative decoder. We then proceed to relate SAD sets to both PD sets and to the stopping 
redundancy hierarchy of a code. 

Definition 5.3: Let H be a parity-check matrix of an error-correcting code C with minimum distance d. A SAD(if) set of 

H is the smallest set S of automorphisms of C such that every 6-set of coordinate positions, 1 < b < d — 1, is moved by 

at least one member of S into a set of positions that do not correspond to a stopping set of H. Similarly, if s < d — 1, an 

s-SAD(if)-set is the smallest set of automorphisms of C such that every 6-set of coordinate positions, for all b < s, is moved 

by at least one member of S into positions that do not correspond to a stopping set in H. Without loss of generality, we 

assume that an s-SAD(Jf )-set contains the identity permutation. For a given code C, we also define 

S:(C)= min |s-SAD(fT(C))|, 

H(C) 

S*(C) - min |SAD(ff(C))|, 
and refer to S*(C) and S*(C) as to the s-automorphism redundancy and automorphism redundancy of C. 

For a given code and parameter s, an s-SAD set may not exist. This is a consequence of the fact that there may be no 
automorphisms that move all arbitrary collections of not more than s coordinates into positions that do not correspond to a 
stopping set in one given parity-check matrix. But whenever such sets exist, they can be related to the stopping redundancy 
hierarchy and PD sets of the code. First, it is straightforward to show that for all 1 < s < d — 1, one has S*(C) < |s-PD(C)|, 
whenever such sets exist. This follows from considering parity-check matrices in systematic form, and from the Singleton 
bound, which asserts that for any linear code d — 1 < n — k. Furthermore, it is straightforward to see that for a restricted set 
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of parity-check matrices, automorphism group decoders trade redundant rows with automorphisms. This is formally described 
by the following lemma, the proof of which is straightforward and hence omitted. 
Lemma 5.4: Let C be an [n,k,d\ code. Then 

p s+ i(C)< (n-fc) x S*(C), (14) 

for all 1 < s < d — 1, provided that an s-SAD(C) set exists. 

One class of codes for which it is straightforward to prove the existence of certain SAD sets is the class of codes with 

transitive automorphism groups, described below. 

Definition 5.5: A group V of permutations of the symbols {0, 1, . . . , n — 1} is transitive if for any two symbols i, j there 

exists a permutation n e T such that in = j. A group is said to be t-fold transitive if for any two collections of t distinct 

numbers i\, . . . , i t e {0, . . . , n — 1} and ji, . ■ . ,jt € {0, . . . , n — 1}, there exists a ir E V such that i\ it = j\, . . . , it n = it- 
Lemma 5.6: Let C be a code with an s-transitive automorphism group. Then there exist b-SAD sets of C for all b < s. 
Proof: Let H be of the form [A\I]. Clearly, the positions of H indexed by k through n — 1 are free of stopping 

sets of size s < n — k. Since the automorphism group of C is ,s-transitive, any collection of not more than s coordinates in 

{0, . . . , n — 1} is moved by some element of Aut(C) into the positions indexed by k to n— 1. Consequently, the automorphism 

group itself represents a (possibly non-minimal) s-SAD set. ■ 
Finding SAD sets of codes is a very complicated task, so that we focus our attention on deriving bounds on the size of such 

sets for specific examples of codes. 

Example 5.1: Consider the [24, 12, 8] extended Golay code £ 2 4- Since the automorphism group of the extended Golay code 

is 5-fold transitive, a 5-SAD set exists, and 85(^24) < I-M24I, where \M.2a\ denotes the order of the Mathieu group A4 2 4- It 

is well known that |A4 24 | = 244823040. 

One can actually show a much stronger result, described below. 

Theorem 5.7: The automorphism redundancy of the £24 code is upper bounded by 14, i.e. 87(^24) < 14. 
Proof: The proof of the result is constructive. Below, we list the particular form of H used to meet the claimed result, 



as well as the corresponding set of SAD automorphisms. The matrix in question is H{Q 2 i) = [Ii2\M], where 

(15) 



M 



h A A 2 A 4 

A h A 4 A 2 

A 2 A 4 h A 

A 4 A 2 A h 



A 



1 1 1 
1 
1 1 



The automorphisms are of the form 9 % x ipi , i = 0, 1, j = 0, 1, . . . , 5, 6, with 
9 = (0, 12)(1, 13)(2, 14)(3, 15) . . . (10, 22)(11, 23); 

i> = (3, 6, 15, 9, 21, 18, 12)(4, 7, 16, 10, 22, 19, 13)(5, 8, 17, 11, 23, 20, 14); 
Both permutations are listed in standard cycle form. Note that these automorphisms include the identity permutation for 

i = j = 0. Incidentally, this is the same matrix that was studied in [25], where it was used to show that PD(£ 2 4)=14. ■ 
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Henceforth, the matrix obtained by combining the H (Q24) -images of all the listed automorphisms is denoted by J?w- The 
subscript W refers to the name of the first author to have studied this matrix - Wolfmann [25]. 

Equation (fl4] i allows one to bound Pq{Q2a) < 12 x 14 = 168, which is significantly larger than the constructive bound 
provided by the matrix in Equation ( fTol l below, which consists of 21 rows only. The matrix was constructed by identifying the 
cog of the [23, 12, 7] code that leads to optimal stopping distance properties, generating a m — 21 cyclic parity-check matrix, 
and then adding a single (fixed) parity-check position to each of the rows. 
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For the extended Golay code, one can show an even stronger result which allows for achieving decoding performance close 
to that of maximum likelihood (ML) decoders. This result will be discussed in detail in the next section. 

In this setting, the parity-check matrix of the code J? [24, 12].* is of the form shown below, and the SAD set consists of a 
set of 23 automorphisms e, r, t 2 , . . . , t 22 , where e denotes the identity element of S24 (the symmetric group of order 24), and 
t = (0 1 2 ... 21 22)(23). The matrix JT[24,i2],* consists of a subset of distinct cogs of the extended Golay code, which 
allows for covering all parity-check equations in the orbits of the chosen cogs. 

It can be shown that for this particular combination of parity-check matrix and SAD set, all uncorrectable erasure patterns 
of weight up to 11 correspond to codewords of the code (see Section VII for more details). 
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We conclude this brief overview by pointing out that there exists a strong connection between the problem of set coverings [26] 
and the problem of finding PD and SAD sets. This relationship can be used to develop simple heuristic search strategies for 
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parity-check matrices that have good SAD-set properties. 

For this purpose, assume that the parity-check matrix is in systematic form, with parity positions confined to the coordinates 
k, ... ,n—l. The number of automorphisms Na needed to move any erasure patterns of cardinality < d— 1 into the last n— k 
positions (free of stopping sets of any size) has to satisfy 
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n — 
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-dn 


-2 
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k - 


1 " 


k 




-2 
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which follows from a result by Schoenheim [26], who derived it in the context of set coverings. 

The crux of the heuristic SAD-set search approach lies in identifying subsets of columns of H that have cardinality larger 
than n — k and that are free of stopping sets of size up to and including d — 1, and in combining these results with ideas 
borrowed from set covering theory. 

For example, it is straightforward to show that the first 15 columns of the parity-check matrix in Equation ( TT3T > are free 
of stopping sets of size less than eight, and that any collection of more than 15 columns must contain a stopping set of size 
smaller than eight. Formula (T% shows that in this case, the smallest number of automorphisms required to map any set of 
not more than seven coordinates of a codeword into the first 15 positions is at least 60 (the same bound, for 12 positions only, 
equals 498). Clearly, there is no guarantee that there exist sufficiently many automorphisms that map arbitrary collections of 
coordinates to these positions. Nevertheless, extensive computer simulations reveal that a good strategy for identifying small 
SAD sets is to use a parity-check matrix in systematic form, to find a large collection of columns K free of stopping sets of 
size less than d, and then perform a search for automorphisms that map subsets of positions in / — K to positions in K. This 
method usually produces good results when the underlying codes have a large automorphism group. 

VI. Automorphism Group Decoders for the BEC 

In order to distinguish between iterative decoders that use automorphisms to reduce errors due to stopping sets and standard 
permutation decoders, we refer to the former as automorphism group decoders. Automorphism group decoders offer one 
significant advantage over iterative decoders operating on redundant parity-check matrices: they have low hardware complexity 
(since only n — k rows of the parity-check matrix are stored, along with very few permutations) and at the same time excellent 
decoding performance. This is, to a certain degree, offset by the slightly increased computational complexity of the decoders. 
The results presented in Section IVIII show that these additional expenses are negligible for error rates of interest. 

We describe next automorphism group decoders (AGD) for cyclic codes, and also provide an example pertaining to extended 
cyclic codes. We restrict our attention to these classes of codes since large subgroups of the automorphism group of such codes 
are known and since the implementation complexity of AGD decoders in this case is very small. Nevertheless, the described 
decoding techniques can be applied to other classes of codes for which some information about the automorphism group is 
available. For the case of extended cyclic codes, it is tacitly assumed that the overall parity-check bit is confined to the last 
position of the codewords and that its index is n — 1. 

Throughout the section, we make use of the following result. 
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Theorem 6.1: ([11, Ch. 8]) Let c = (co, c\, . . . , c n _i) be a codeword of an [n, k, d) cyclic code. The automorphism group 
of the code contains the following two sets of permutations, denoted by C\ and Ci'- 
C\. The set of cyclic permutations 7 , 7 1 , . . . , 7™ -1 , where 7 : i — ► i + 1 modn; 

C2: The set of permutations £°, C 1 , ■ ■ ■ , C c-1 > where £ : i — > 2 • i mod n, and where c denotes the cardinality of the cyclotomic 
coset (of the n-th roots of unity) that contains the element one. 

For extended cyclic codes, we use the same notation C\ and C2 to describe permutations that fix c n -\ and act on the 
remaining coordinates as described in the theorem above. All automorphisms can be decomposed into products of disjoint 
cycles. Permutations in C\ have one single cycle (or two cycles, for the case of extended cyclic codes), while permutations in 
C2 have a number of cycles that equals the number of cyclotomic cosets r of the n-roots of unity (or r + 1 cycles, for the case 
of extended cyclic codes). The number of cycles in the automorphisms used for decoding influences the hardware complexity 
of the scheme and should be kept small. 

AGDa Decoders: These decoders use permutations drawn from the set C\, which reduces the permuter architecture to one 
single shift register. One way to perform automorphism group decoding is to set the permuter to 7 , until the presence of a 
stopping set is detected. In that case, the AGD^ decoder applies a randomly chosen cyclic shift 7* to the current word, i 7^ 0. 
If the iterative decoder encounters another stopping set, the whole process is repeated with a (yet another) randomly chosen 
cyclic permutation. The decoding process terminates if either all permutations in C\ are tested or if the decoder successfully 
recovers the codeword. 

Assume that the number of cogs of a cyclic code is at least n — k. In this case, the parity-check matrix used for decoding 
consists of n — k different cogs, provided that such a matrix has full row-rank. The redundant parity-check matrix consisting 
of the collection of all vectors in the C\ -orbits of the cogs is henceforth denoted by J?agd a - 

AGDb Decoders: These decoders use permutations drawn from both C\ and C2, so that the resulting permuter architecture 
is slightly more complex than that of AGD^ decoders. If a stopping set is encountered, the decoder first tries to resolve 
this set by applying a randomly chosen permutation from C\. Only after the whole set C\ is exhausted, a randomly chosen 
permutation from C2 is applied to the current decoder output. The parity-check matrix used for this decoder is generated 
in terms of a greedy procedure. First, an arbitrary cog is chosen. Then, another cog that is not in the orbit of the first cog 
under any permutation in C2 is chosen, if such a cog exists. The procedure is repeated until either the set of cyclic orbit 
generators is exhausted or until the matrix contains n — k rows. In the former case, additional rows are chosen from the set 
of second-smallest weight codewords of the dual code, provided that they give rise to a matrix of row-rank n — k. 

Figure |l(a)| and Figure |l(b)| illustrate the permutation operations used in the described decoding architectures. Here, the 
check nodes and edges in the graph of an extended cyclic code are fixed, while the variable nodes are permuted according to 
the permutations C\ and C2 defined in Theorem 16. II 

The set of minimum weight cogs can be partitioned into cog families (CF), where all members of one given family have 
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(a) d 

Fig. 1. Permutations of C\ and C 2 for extended cyclic codes. 



(b) C 2 



TABLE VIII 

Intersection numbers of the CFi and CF 2 families of the [23, 12, 7] Golay code. 



CFi 
k e 


Intersection numbers 
\oo K \ \OZ K \ \ZO K \ \ZZ K \ 


*i, |*i| = 2 

*2, *2 = 8 
*3, |*3l = 12 


8 8 7 
4 4 4 11 

2 6 6 9 




CF 2 

ft e 


Intersection numbers 

\oo K \ \OZ K \ \ZO K \ \ZZ K \ 


*4, |*4| = 6 
*4, |*4| = 16 


4 4 4 11 

2 6 6 9 



identical intersection numbers. More precisely, a CF is a set of cogs with identical intersection cardinalities n £ CT ,j|> and 
consequently, identical values of SW.2, as given in Equation (l3[ . This result is a consequence of the fact that cogs within the 
same family have the same parameters |XY| K , introduced in Lemma |431 

Cogs from the family with optimal intersection numbers are good candidates for constructing parity-check matrices amenable 
for automorphism group decoding. In this case, it is important to select subsets of cogs that are not images of each other under 
the automorphisms used by the decoder. For example, for the [23, 12, 7] Golay code, the 22 cogs of minimum weight eight 
can be grouped into two families CF\ and CF-x, each consisting of 11 cogs. Both families are closed under the permutations 
in C2. 

We partition the set of shift indices k, k E {1, 2, . . . , n — 1}, into subsets ^ which have equal parameters |OO k |, \OZ K \, 
\ZO K \, and \ZZ K \. Table [Villi lists a selected subset of intersection numbers for cogs in the CF\ and CF 2 families. It can be 
seen from the table that every cog in CF\ has parameters |OO k | = 0, \OZ K \ = \ZO R \ = 8, and \ZZ K \ = 7 for the two shift 
indices forming 

For the [31, 16, 7] BCH code, the set of 15 cogs of weight eight forms one single family CF. A selected set of intersection 
numbers of this family is shown in Table [IX] 

TABLE IX 

Intersection numbers of the unique CF of the [31, 16, 7] BCH code. 



CF 
ft G 


Intersection numbers 

\oo K \ \OZ K \ \ZO K \ \ZZ K \ 


#\ I* 1 ! = 2 

* 2 , |* 2 | = 4 
* 3 , |* 3 = 24 


4 4 4 19 
8 8 15 
2 6 6 17 
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VII. Performance Results 

We present next a selected set of simulation results for the bit error rate (BER) and frame error rate (FER) performance of 
both redundant parity-check matrices and various automorphism group decoders. The BER equals half of the fraction of the 
residual erasures, as at the end of the decoding procedure one can guess all uncorrectable bits. A frame error is declared if at 
least one symbol in the frame estimated by the decoder does not match the corresponding symbol in the transmitted frame. 
Our results indicate that the FER performance of codes operating in the above described manner can be improved by adding 
an additional "guessing" feature, which is described in more details in the concluding part of the section. 

A. The Golay and Extended Golay Code 

The residual bit error rate performance of edge-removal decoding on the cyclic parity-check matrix representation based on 
cog 23 12 A i s shown in Figure [2(a)] In this figure, the cyclic 23 x 23 matrix is truncated to 11, 16, 18, and 23 rows, corresponding 
to the upper bounds on the stopping redundancy hierarchy obtained from this cyclic representation. For comparison, the 
performance of the ML decoder is also shown in the same figure. 

In Table IXl we list the number of uncorrectable erasure patterns of size up to a = 12 in several parity-check matrices of 
the [24, 12, 8] extended Golay code; J? [24. 12],* an d -f^w were defined in Section|V] while the matrix i?ns corresponds to the 
matrix of dimension 34 x 24 described in [4]. The index ML refers to the erasure patterns that cannot be decoded by an ML 
decoder. Figures |2(b)| |2(c)| and |2(d)| show the performance of iterative decoders operating on a selection of the parity-check 
matrices in Table IXl using standard BP decoding on #"[24,12],* an d using AGD^ and AGD^ decoders. 

As can be seen, there is a significant performance gain of AGD^ or AGD^ decoders when compared to that of standard 
edge-removal decoders operating on the redundant parity-check matrix described in [4] or on any other non-redundant Tanner 
graph. In fact, the BER performance of AGD^ and AGD^ decoders approaches the performance of ML decoders. For 
EP < 0.15, all matrix representations require an almost identical average number of iterations, indicated by the vertical bar in 
Figures [2(b)] gg)) and [2(d)! 



TABLE X 

Uncorrectable erasure patterns of the [24, 12, 8] extended Golay code. 



cr 


Number of uncorrectable erasure patterns 


^[24,12],* 011 BP 


^[24,121,* on AGDa 


H w on BP 


H HS [4] on BP 


ML 
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190 
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2231 
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15881 
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79381 
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293703 


759 


759 


3284 


759 
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805556 


12144 


12158 


78218 


12144 


10 


1613613 


91080 


93477 


580166 


91080 


11 


2378038 


425040 


481764 


1734967 


425040 


12 


2690112 


1322178 


1547590 


2569618 


1313116 


> 13 
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O 
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O 
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0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 



<- EP <- EP 

(b) (d) 

Fig. 2. Performance of the [23, 11,7] and [24, 12,8] (extended) Golay codes. 



B. BCH Codes 

Figure [3] plots the performance of iterative edge-removal and automorphism group decoders with and without permutation 
features, operating on several (redundant) parity-check matrices of the [127, 113, 5] BCH code. 

The standard edge-removal decoder uses parity-check matrices of cyclic form, generated by the cog cog m 113 a- The 
matrices have m = n — k = 14 and m = 34 rows, respectively. The cyclic parity-check matrix with 36 rows offers significant 
performance improvements, when compared to its counterpart with 14 rows. Excellent BERs can also be achieved with the 
AGD^ decoder that operates on a parity-check matrix containing 14 rows of distinct cogs. 

Figure [3] shows that the AGD^ decoder performs very close to the ML union bound, which is known to be fairly tight for 
the BERs considered. Again, a vertical bar (placed at EP = 0.03) is used to indicate the BER region for which the average 
number of iterations needed for successful decoding is approximately the same for all tested decoders. 

Figures |4(a)| |4(b)| and |4(c)| show the performance of several classes of iterative decoders operating on the [31, 16, 7] BCH 
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o Various cogs, m — 14, BP 

* <*>g 127 ,|i3.A» m - 1 1 ' BP 

: -^^cogiaT.iis.A' 771 = 34 ' BP 
— * — Var. cogs, m — 14, AGDa 
- a - ML Union Bound 

i i i: 1 1 ! q- i 1 1 1 1 1 s ^ 

0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.08 0.07 0.06 0.05 0.04 0.03 0.02 

<- EP <- EP 

Fig. 3. Performance of the [127, 113, 5]-BCH code. 

code. The AGD A decoder uses a parity-check matrix that consists of all 15 cogs of the code, and for comparison, standard 
iterative decoding is performed on the same matrix. In addition, the performance of standard iterative decoders on three different 
cyclic parity-check matrices with m = n — k = 15, m = 21, and m = 31 rows is plotted as well. The generating cog of these 
matrices is cog 31 16 A . 

Performance results for the BCH code under consideration are presented in Figure |4(d)| pertaining to the cyclic parity-check 
matrices based on cog 31 16 A and cog 31 16 c in Table I VIII In all simulations, the number of rows in the parity-check matrix 
equals the smallest listed value in the table. By considering only the representation indexed by cog 31 16 A , it can be observed 
that the most significant BER improvement is achieved when the matrix has m = 18, rather than m = 15 rows. The results 
also show that for short block codes, like the ones considered in this example, almost all cogs give rise to similar residual 
bit error rates. Nevertheless, for codes of longer length and larger co-dimension, the particular choice of the cog may have a 
significant bearing on the stopping set characteristics. 

In all the examples investigated, the improved performance of automorphism group decoders comes at the cost of increased 
decoding complexity, when compared to edge-removal techniques. Clearly, to correct more erasures than standard iterative 
decoders, automorphism group decoders have to go through additional decoding iterations, and in addition, perform a certain 
number of permutation operations. A measure of computational complexity of automorphism group decoders is plotted in 
Figure [5] Here, the number of excess iterations (when compared to edge-removal decoding operating on the same channel 
output) is plotted versus the EP of the BEC. As one can see, at most four additional iterations suffice to resolve the decoding 
ambiguities of most error patterns that are uncorrectable by belief-propagation techniques. 

The results in Figures [4(a)| and [4(b)| show that there exists a significant gap between the performance of automorphism group 
decoders and ML decoders when considering FERs, rather than BERs. 

To mitigate this problem, we propose to include an additional feature into the edge-removal and automorphism group decoding 
algorithms. The feature in question is guessing: when the decoder fails to correct a given erasure, it continues decoding by 
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Various cogs, m — 15, BP 
c °g31,16,A' m = 15 > BP 
C °S31.16,A' m = 21, BP 
c °g31,16,A> m = 31, BP 

Var. cogs, m — 15, AGD ^ 




0.25 0.2 0.15 0.1 

<- EP 

(c) 



^^ co S31,16,A' m = 15 . BP 
^^ C °S31,16,A' m = lg . BP 
^^ c °g31,16,A. m = 19. BP 
^^ co S31.16,A. m = 21 . BP 
-*- C °S31,16,C. m = 15 > BP 

-0- cog 31 16>c , m = 21, BP 
e - ML estimation 




Fig. 4. Performance of the [31, 16, 7]-BCH code. 



guessing the value of one of the erased bits. The idea of incorporating guessing methods into belief propagation algorithms was 
first described in [27] and, in the context of stopping set analysis in unpublished work by Han and Siegel [28]. Figure [6] shows 
the FER performance improvement for the [31, 16, 7] BCH code achieved through combined automorphism group decoding 
and bit guessing. As it can be observed, the FER performance of AGD^ decoders with the guessing feature approaches the 
performance of ML decoding. 

VIII. Conclusions 

We introduced the notion of the stopping redundancy hierarchy of linear block codes, and derived lower and upper bounds 
on the elements of this ordered list. We also investigated the stopping redundancy of cyclic-parity check matrices, and proposed 
new redundant parity-check matrix constructions and automorphism group decoding techniques. In this setting, we introduced 
the notion of a stopping set automorphism group decoding set, and related this new code invariant to the stopping redundancy 
hierarchy. 
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Various cogs, m — 14, BP 




1 - 



p ' 1 1 1 ' 1 

0.08 0.07 0.06 0.05 0.04 0.03 0.02 

<- EP 

Fig. 5. Number of iterations of the automorphism group decoder for the [127, 113, 5] BCH code. 




0.25 0.2 0.15 0.1 0.05 

<- EP 



Fig. 6. Performance improvements achievable by the guessing method. 



Appendix I 
Proof of Theorem 13. 8 1 

We start the proof by introducing the notion of an orthogonal array [7]. 

Definition LI: ([29, p. 5]) An orthogonal array of strength t is a matrix oftnxn elements with the property that every mxt 
subarray contains each possible i-tuple the same number of times. The codewords of an [n,k,d] code C form an orthogonal 
array of dimension 2 fe x n and strength d 1 - — 1, where, as before, d 1 - denotes the dual distance of C. 
It can be shown that the set of all codewords of a code forms an orthogonal array of strength d 1 - - 1 [29]. 

Definition 1.2: Let C be an [n, k,d] code and C its dual code. The ensemble Mc( m ) is defined as the set of all m x n 
matrices with rows chosen uniformly, independently, and with replacement, from the set of 2 n ~ fc codewords of C . 
Let A be the set of all binary t-tuples of weight one. Let Ei be the event that the z-th collection of t columns in an array 
randomly drawn from the A4c(m) ensemble contains no element from A, i.e. that the i-th collection of columns corresponds 
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to a stopping set of size t. If {S(t) = 0} denotes the event that a given array does not contain stopping sets of size t, then 
p {f] Ei} = p {S(t) = 0}. Observe next that 

P{Ei\ = P { P| {fc th row of the i lh restriction £ A} } 

k=l 

= ( P { 1 st row of the i th restriction $ A})™ (L18) 
= (l-i-2"*) m , 

where the last step follows from the fact that the occurrence of every t-tuple in the orthogonal array is equally likely, with 
probability 2 _t . The dependence number r of the events Ei, defined in Equation (13 .7t . equals 

fn\ fn — t\ 

(r+l)= J- , . (1.19) 



The above result is a consequence of the fact that two collections of t columns are co-dependent if and only if they share at 
least one column. If one collection of t columns is fixed, another collection is independent of it if its columns are chosen from 
the remaining n — t columns. 

By inserting the expressions (II- 1 81 > and (II. 19b into the defining inequality of LLL gives the following sufficient condition 
for the existence of a parity-check matrix with m rows, free of stopping sets of size t: 

•c- 2 - ')"((;) -(";')) sl - 

This implies that, if m satisfies the condition above, there exists at least one parity-check matrix of dimension m free of 
stopping sets of size t. Consequently, this bound on m represents an upper bound on pi{C). 

A sufficient condition for the existence of a matrix with m rows that does not contain stopping sets of size j £ {1,2,..., t}, 

is 

n \ I n — j 



By observing that 



max fl-i-2- J ) = h-t-2' 

we arrive at 

' n \ ( n — j 



'-«TE((;j-[71]^' cum 



Note that Equation dl.20| i can be rewritten as 



m > 



i + i°gE(G)-C7) 



In order to ensure that the chosen matrix has full row-rank, it may be necessary to add n — k — £ + 1 additional rows. 

Observe that any collection of at most d — 1 columns of a parity-check matrix is linearly independent. This follows from the 
fact that the minimum distance of the code is d. As a consequence, in order for two collections of t columns to be independent, 
one must have 2 • t < d. This restricts the validity of the above bound to stopping distances £ < [^^-\ only. Setting t = £ — 1 
completes the proof. 
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Appendix II 

Asymptotic Behavior of the Bound in Theorem 13. 8 1 



We investigate the asymptotic behavior of the bound in Theorem 13.81 Let mf be the first summand of Eq. (O, i.e. 

i + iogE((p-C7)) 



(11.21) 



-log(l-M) 

Let us consider the case £ — 1 = An, for < A < 1. In this case, the logarithm in the numerator grows at most linearly with 
n, since 

E Cfj < 2", (n.22) 

3=1 W 

and the second sum under the logarithm never exceeds the first. 

As a result, the asymptotic behavior of the given upper bound is dominated by the expression in the denominator, which, 
for large n, takes the form 

log 1 



2t-l J ~ 2 A ' 
and therefore 

, (l + n)2 An 2 A " 

m ~ ~ — . 

An A 

Consequently, for codes with minimum distance d = const. • n, the upper bound on m is exponential in n. 

Appendix III 
Proof of Theorem|3.9I 



Theorem |3.9| can be proved using arguments similar to those described in the proof of Theorem 13.81 First, observe that the 
number of distinct collections of columns, and consequently, the number of events Ei used in LLL equals 



i- - 1 

Since the dependency number for the high probability variation of Lovasz Local Lemma is the same as the one given in 
Theorem 13.81 it follows that 

/ \ e[(?)-(V)-i] 

fi-t. 2 -*r<=^=rii- ' 1 



After some simple algebraic manipulation, one can show that the above expression gives rise to the following bound: 



log— ^+ e 



n-j\ _ i 



log 1 - 



m > -. — . 

log(l-^) 

As a result, for a given set of code parameters, with probability greater than 1 — e every parity-check matrix with m rows has 
stopping distance at least t + 1. Substituting t = £ — 1 completes the proof. 
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Appendix IV 
Cyclic Parity-Check Matrices of QR Codes 

Let Q be a QR code of prime length n — 3 ( mod 4), and let TV and Q denote the set of quadratic non-residues and 
residues of the underlying finite field, respectively. The idempotent of the code is the polynomial J2i< n fi x% ^ where /j = 1 if 
i E N U {0}, and is zero otherwise. The idempotent can be used as the first row of a (redundant) cyclic parity-check matrix 
of Q. 

In what follows, we assume for two fixed indices i and j that k = i — j (modn). For a prime n = At — 1, \Q\ = 2t — 1, 
and for each integer k e Q, k / ( mod n), there exist exactly t — 1 ordered pairs (i, j), i ^ j, such that i, j G Q. Similarly, 
for fixed k and i E Q, there exist t — 1 distinct values j E N that result in the given value of k. 

The lower bound of Example 14.11 can be derived as follows. First, observe that the number of overlapping zeros between 
the first row and its n-th cyclic shift, \ZZ K \, equals 

n-1 

\ZZ K \ = 2(1 - - /i_ K ). 

8=0 

Let us now determine the number of pairs i,j € Q, for which k G N. As already pointed out, for each k G Q, there 

are t—1 choices for pairs i,j G Q, and t—1 choices for i G Q, j G N. The (2t — l) 2 pairs i,j G Q arise from (2t — l)(t— 1) 
different values of K, and there are (2t — 1) choices for k = that lead to i e Q. In addition, (2t — l)(t — 1) values of k 
result in i, j E Q, k E N, while for t—1 values of k € JV one has i, j G Q. Therefore, the number of pairs such that 
i, j G Q equals t — 1 for k G Q, as well as for k E N. As there are f — 1 quadratic residues, for k E Q as well as for k E N, 
it follows that |i?Z K | = t - 1 (excluding k = 0). 

The row-weight of the cyclic parity-check matrix equals co — \N\ + 1 = 2t, where co = \OO k \ + \OZ K \ = \OO k \ + \ZO K \. 
Therefore, \OZ K \ = \ZO K \. Furthermore, one has |00 K | + \OZ K \ + \ ZO K \ + \ZZ K \ = n, and the same result can be deduced 
from the fact that 

n— 1 n— 1 p— 1 

\OZ K \ = M 1 - fi-n) - E & ~ E = 2 ^ \°°*\- 

i=0 i=0 i=0 

One can now derive an expression for |OO k | as follows: 

\00 K \ = n-|ZZ H |-|OZ K |-|ZO„| = n-(t-l)-2.|OZ„|=n-i + l-2-(2i-|OO lt |) 

= n-i + l-4f + 2- 100«|, i.e. 
\OO k \ = 5*- 1 -n = 5i- 1 - (4i- 1) =t, 

so that 

|OZ K | = \ZO K \ =2t- \OO k \ =2t-(t-2)=t. 

Since the intersection numbers do not depend on the value of k and on this number being a quadratic residue or non-residue, 
we henceforth omit the subscript k. Also, note that we do not make use of the intersection numbers for which k = 0, since 
we are interested in counting the number of stopping sets resolved by distinct rows of a (redundant) parity-check matrix. 
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Inserting the expressions above into the formula of Lemma 14.51 shows 

\ZZ[ 



|£^n£^ +K | = \OQ\ 



\zz\ 

(T — 1 



\oz\-\zo\- 



cr-2 



n + I ( 2f2 
a - 1 



71 + 1 



2 / n-3 



a-2' 



so that 



m— 1 m—fi 



Scr,2 — I^CT.jLt n Ect^i+kI 

— 1 K— 1 



m \ 71+1 
2 



n— 3 
4 

CT- 1 



n + 1 / ^ 
4 U-2 



It is also straightforward to show that 



2 

£<r,l| V" (m - «)|£<r,l n S CT 

K — 1 

n + 1 



m— 1 



71 + 1 



2 

CT - 1 



mod ■ m 1 



-(m — 1) 



(n + 1)/^ 



cr - 1 



(j — 2 



From the condition | U™ 1 E CT) <| > (™) and <r = £-1, we obtain a lower bound for the number of rows that a parity-check 
matrix consisting of cyclic shifts of the idempotent must have in order to have stopping distance at least I: 

( n ) - M 



u,£ > max 



(IV.23) 



In the above equation, we used 



M = 



n+1 



n— 3 
4 

a - 1 



(« + 1) / 4 

v cr - 2 



Note that the set of quadratic residues also forms a cyclic difference set with parameters (n, fc, A) = (n, ^^r^"; ^V^) ^0], 



so that the results of Section IV can be used in this context as well. 



It is also of interest to find an upper bound on the stopping distance i of a redundant n x n parity-check matrix that consists 
of all cyclic shifts of the idempotent of a QR code. For this purpose, we use Equation ( IIV.23I ) with m = n, and ask for the 
largest value of a for which 

We point out that although the value a + 1 serves as an upper bound on the stopping distance of the parity-check matrix under 
consideration, there is no guarantee that the matrix itself has a stopping distance that meets this bound. 

To this end, we only seek a solution for I = a, since it can be shown that this value of the parameter imposes the tightest 
restriction on the inequality. 

Also, for both n and a sufficiently large (where a = o(n)), we can approximate the above inequality by 



since 




(IV.24) 
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and 



n 2 ( ( n/4 \ n ( n/4 \\ 2 /n/4 

T vU-lJ + 4 U-2/J I -t 



As the family of QR codes contains an infinite number of codes, we provide next an asymptotic analysis that allows us to 
upper bound the achievable stopping distance of redundant parity-check matrices based on the idempotent of the code. 

To find an asymptotic lower bound for the stopping redundancy of cyclic parity-check matrices, we use the bound [11, p. 
309, Eq. 16] 



where (3n is an integer, H(0) — —/3 In /3—{l—/3) ln(l— 0) stands for Shannon's entropy in natural units, /i(/3, n) — , 1 , 

y8n/3(l— (3) 

and fzid.n) = — ; . 

For j3 — > 0, the following asymptotic formula holds for H((3): 

H((3) = -plnp- (l-/3)ln(l-/3) 
= -l3\nP-f3 + f3 2 /2 + 0(f3 3 ). 

In the same asymptotic domain, n) and /2(/3,n) take the form 

h(J3, n) = ) = — L= (1 + 0{fl)) , 

and 

/ 2 (/3,n)= 1 = _L- (1 + 0(0)). 

y/2im/3(l - (3) V 2 n n (3 

Denote the lower bound for the left hand side of Equation ( II V. 241 ) by L(n, a). Similarly, let the upper bound for the two terms 
on the right hand side of the same equation be denoted by Ri(n,a) and R,2(n, a), respectively. 
Invoking the asymptotic expressions described above shows that 

(") ~ Vf=fj*T e-^V(2»H°<-V»') { l + (a/n)) = L(n,a), 



and that 



n a 



— no 2 



M 2 \ 1 e -2 ff +2.V«+o(^/n 2 ) (i + 0((r/n))= J Ri(n,(r), 

V o J V4vr2 4 -a 4CT + 1 

( n/4 ) <-n ia+1 a 2 , 1 — e- 4CT + 8CT2 /"+°^ 3 /" 2 ) (1 + 0(a/n)) = R 2 (n, a). 



Tedious, but straightforward algebraic manipulation reveals that the above inequalities imply 

1 < v^o^ 1 p( - CT+3 ^/ 2 n) n^ +1 a 2 p( _ 3g+15(T 2 /2n) 

The square-root bound for the minimum distance of QR codes asserts that their minimum distance is lower bounded by 
/n. Although the minimum distance of many quadratic residue codes exceeds this bound, little is known about the asymptotic 
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behavior of the exact minimum distance. We henceforth define the designed minimum distance, which equals ypn, and proceed 
to evaluate the right hand side of the expression in the above equation for a = \fn. In this case, the resulting formula equals 

Ci n v/ ™ +3 / 2 e -Vn-2V™ In 2 „ C2 ^3^+2 e ~3^/n-iy/n In 4 

for some positive constants c\ and c-i- For n — ► oo, this expression diverges to — oo. As a consequence, the stopping distance 
of redundant cyclic parity-check matrices generated by the idempotent of a QR code cannot meet the square-root bound. 
Furthermore, no positive fraction of this bound is attainable by a matrix of this form. 

This finding implies that, rather than using idempotents as generators, cogs should be used instead, since they have 
significantly lower density. 

Appendix V 

Construction of Redundant Parity-Check Matrices: Double-Error Correcting BCH Codes 

We describe the generalized Hollmann-Tolhuizen (HT) method for generating redundant parity-check matrices of double- 
error correcting BCH codes. The procedure consists of four steps. In the first step, the algorithm outlined in [10] is used to 
create a redundant parity-check matrix for the Hamming supercode. This matrix is used as the first sub-block in the parity-check 
matrix of the BCH code. In the second step, a second sub-block of rows is added, consisting of codewords of the dual of 
the BCH code that represent a basis of the code. The third step consist of adding redundant rows that resolve stopping sets 
corresponding to codewords of weight three and a subclass of codewords of weight four in the Hamming code. The fourth step 
introduces into the parity-check matrix a set of redundant rows obtained by a greedy search strategy, designed to completely 
resolve stopping sets of size four. 

The generalized HT algorithm is summarized below. 

Step 1) A redundant parity-check matrix for a Hamming code that resolves all stopping sets that correspond to correctable 
erasure patterns of size up to (but not including) a = 4 is created first. To this end, a generic (m, er) erasure correcting 
set with fa — \o<g 2 (n + 1) and a = 4 is multiplied with the standard parity-check matrix of the Hamming code. 
The resulting matrix contains only unresolved stopping sets of size a = 3 and a = 4, which either correspond to a 
codeword of the Hamming code or contain the support of one such codeword in their support. We refer to this matrix 
as a generic erasure correcting matrix, and denote it by ffh.g- Figure [7] visualizes the first step of the construction 
procedure. 

Step 2) The matrix obtained in Step 1 is augmented to form a parity-check matrix of a BCH-code with minimum distance 
d = 5. This is achieved by adding the binary expansion of the row vector 

^bch = ( a°' (1+c) a 1 -^ ... a (»-i)(i+c) ) , 

where c is chosen according to the standard procedure for generating BCH codes (one common choice being c = 2). 
Here, a is as defined in Section IIV-C.2I We refer to this second component of the redundant parity-check matrix of 
the BCH code as Hbch- Figure [8] visualizes the second step of the construction procedure. 
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Step 1 



h, g | 



Fig. 7. Step 1: Transform H b s into a generic (m,<r) erasure correcting set Wh, g- 



Step 2 

Step 3 
Step 4 



Fig. 8. Steps 2-4: Append JJbch (Step 2), and add code-defining parity-check equations to H h g (Steps 3 and 4). 

Step 3) In this step, an additional set of redundant rows is added to the matrix in order to resolve all stopping sets of size 
cr = 3 and a subset of those of weight a = 4. To this end, we use the fact that all unresolved stopping sets of size 
three and four correspond to codewords of the Hamming code. 

Let t denote the support of an unresolved stopping set of size a = 3. The restriction of -Hh, g to t equals 

[(«") (« 6 ) K + a 6 )] 
with < a, b < n — 2, a a ^ a b , whereas the restriction of i?BCH to t equals 

[(^) ( a * b ) (K+a b ) 3 )], 

where we have set c = 2. 

Consider first all stopping sets for which a 3a — a 3b , but a a ^ a b . 

Lemma V.l: In a finite field with characteristic two and order n + 1, for any primitive element a, a 3a = a 3b implies 
(a a + a b ) 3 = a 3a iff a a ^ a b . 

Proof: If a a = a b , the lemma does not hold, as (a a + a b ) 3 = 0. We henceforth consider the case a a ^ a b 
only. Since the field has characteristic two, one can write 



Consequently, 



(a a + a b ) 3 = a 3a + 3a 2a+b + 3a a+2b + a 3b = a 2a+b + a a+2b . 



a 2a+b + a a+2b = ^Sa ^b-a + a a-b\ 



As 3a — 3b = j n, for some integer j > 0, it follows that a — b = jn/3 and b — a = 2 j n/3. In order for a a ^ a b 
to hold, j has to be restricted to j ^ (mod 3). For these values of j, one has (a b ~ a + a a ~ b ) = (a"/ 3 + a 2 "/ 3 ). 
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To complete the proof, we first prove the following more general claim: for each u 6 {2k + 1 | k € N}, it holds that 
Y^tZi a, m t u = 1. The result of the lemma then follows by setting u = 3. 
We prove the claimed result by showing that 



u-l 

in/u 



a 
i=l 



We observe that 



u-l 


(u-l)/2 


u-l 








i=l 


i=l 


i=(u+l)/2 




(«-l)/2 


(u-l)/2 




= J2 « 2m/ " + 


a 2(»'+u/2-l/2)n/« 




»=i 


i'=l 




(tt-l)/2 


(«-l)/2 










i=l 


i'=l 




u-l 






i=i 





where we have introduced i' = i — u/2 + 1/2 and used the fact that a n = 1. This completes the proof. ■ 
Assume that n/3 is an integer. Then there exist exactly n/3 stopping sets corresponding to Hamming codewords of 
weight three for which a 3a = a 3b , as the free parameter a can only be chosen in the range < a < n/3. The variable 
b needs to be chosen accordingly within n/3 < b < n. If we switch to the binary representation of the parity-check 
matrix, it is easy to see that the restriction of i2"h, g to t only contains rows of weight zero or two, whereas the same 
restriction of Hbch only contains rows of weight zero or three. As a result, each stopping set corresponding to a 
weight-three codeword of the Hamming code with a 3a = a 3b , can be resolved by an appropriate linear combination 
of one row from i?h,g an d one row of Hbch- We conduct the search for the appropriate rows in these matrices in 
a greedy fashion, i.e. by eliminating as many stopping sets as possible with each redundant row. Figure [8] visualizes 
this step. 

Step 4) Step 3 gives rise to parity-check matrices that resolve all stopping sets of size a = 3, for which a 3a = a 3b , but 
a a =^ a b , and n/3 is an integer. As an example, this claim is true when n = 63 and the corresponding code is a 
[63, 51, 5] BCH code. In order to resolve stopping sets that do not satisfy these constraints, we use greedy computer 
search techniques to identify a collection of redundant parity-checks suitable for accomplishing this goal. These 
checks are added to the concatenation of -ffh, g> Hbch, and the parity-checks found in Step 3. 

Acknowledgment: The authors are grateful to the anonymous reviewers for their constructive comments that significantly 
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